CLAIMS 



1 1 . Apparatus comprising 

2 a portable electronic device, 

3 a digital camera associated with the portable electronic 

4 device, and 

5 software configured to run on the portable electronic device 

6 and to derive handwriting and control information from hand 

7 motion of a writing instrument in the vicinity of the digital camera. 

1 2. The apparatus of claim 1 in which the portable electronic 

2 device comprises a mobile telephone or a personal digital assistant. 

1 3. The apparatus of claim 1 in which the digital camera is 

2 attached to the portable electronic device. 

1 4. The apparatus of claim 1 in which the digital camera is 

2 incorporated in the portable electronic device. 

1 5. The apparatus of claim 1 also including an infrared filter 

2 arranged to filter light being received from the writing instrument. 

1 6. The apparatus of claim 1 also including a lens arranged to 

2 alter the focal length and/or depth of field of the digital camera. 

1 7. The apparatus of claim 1 also including a mechanism 

2 configured to enable the digital camera to be attached to a writing 

3 surface. 

Attorney Docket 11627-007001 



23 



1 8. The apparatus of claim 7 in which the mechanism 

2 comprises a suction device configured for attachment to a white 

3 board. 

1 9. The apparatus of claim 7 in which the mechanism 

2 comprises a clip configured to grasp paper. 

1 10. The apparatus of claim 7 in which the portable electronic 

2 device includes a writing surface. 

1 11. The apparatus of claim 1 0 in which the writing surface is 

2 on a protective cover. 

1 12. The apparatus of claim 1 in which the software is 

2 configured to define a mapping between a sensor surface in the 

3 digital camera and a space in which the hand motion is occurring. 

1 13. The apparatus of claim 1 in which the software is 

2 configured to define the mapping in response to calibration steps 

3 that include a user marking three locations in the space in which 

4 the hand motion is occurring. 

1 14. The apparatus of claim 1 in which the software is 

2 configured to derive the location and trajectory of the hand motion. 

1 15. The apparatus of claim 1 in which the software is 

2 configured to generate the handwriting and control information 

3 based on processing cycles each associated with one location of the 

4 writing instrument. 

1 16. The apparatus of claim 1 in which the software is 

2 configured to discriminate light received from the writing 
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3 instrument by locking onto a carrier frequency at which light from 

4 writing instrument is modulated. 

1 17. The apparatus of claim 1 in which the software is 

2 configured to determine a tilt of the writing instrument relative to a 

3 direction normal to a writing surface. 

1 18. The apparatus of claim 1 in which the portable electronic 

2 device includes a display and the trajectory of the hand motion is 

3 shown on the display in real-time. 

1 1 9. The apparatus of claim 1 8 in which the display is not 

2 touch-sensitive. 

1 20. The apparatus of claim 1 in which the portable electronic 

2 device includes a digital signal processing chip and a general 

3 purpose microprocessor and the software is ran in part on the chip 

4 and in part on the microprocessor. 

1 21. The apparatus of claim 1 in which the portable electronic 

2 device includes a wireless communication facility and the software 

3 is configured to communicate the handwriting and control 

4 information to a remote location. 

1 22. The apparatus of claim 1 in which the digital camera is 

2 configured to receive light that has been reflected from the writing 

3 instrument. 

Attorney Docket 1 1627-007001 

25 



1 

2 



23. The apparatus of claim 1 in which the digital camera 
comprises a still camera. 



1 24. The apparatus of claim 1 in which the digital camera 

2 comprises a video-capable camera. 

1 25. The apparatus of claim 1 also including an infra-red beacon 

2 configured to be directed at the writing instrument. 

1 26. The apparatus of claim 1 in which the software is 

2 configured to apply pattern recognition to signals from the digital 

3 camera. 

1 27. A method comprising 

2 exposing a sensor of a digital camera to a writing 

3 instrument that is being subjected to hand motion, and 

4 in a device associated with the digital camera, processing 

5 the data to infer handwriting and/or control information based on 

6 the hand motion. 

1 28. The method of claim 27 in which the exposing of the 

2 sensor includes receiving light that originates from light sources on 

3 the writing instrument. 

1 29. The method of claim 27 in which the exposing of the 

2 sensor includes exposing the sensor to a trace or other marking left 

3 by the writing instrument. 

1 30. The method of claim 29 in which the trace or other marking 

2 includes ink selected to increase a signal-to-noise ratio of light 

3 received by the sensor. 
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3 1 . The method of claim 27 in which the exposing of the 
sensor includes processing of an image of a tip of a writing 
instrument. 



1 32. The method of claim 3 1 in which the tip of the writing 

2 instrument is characterized by being of high contrast with the 

3 environment in which it is used. 

1 33. The method of claim 27 in which the sensor is exposed to 

2 hand motion occurring with respect to any arbitrary surface at any 

3 arbitrary angle. 

1 34. The method of claim 27 also including 

2 calibrating the digital camera in a manner to permit 

3 correctly inferring linear hand motions. 

1 35. The method of claim 27 also including calibrating the 

2 processing of data using information derived when the writing 

3 instrument is touched to at least two points on a writing surface. 

1 36. The method of claim 27 also including automatically 

2 switching the digital camera from one format to another by 

3 coupling the camera, or a device to which it is connected, to 

4 another mechanism. 

1 37. The method of claim 27 also including adjusting a tilt 

2 and/or swivel position of the camera for better coverage of a 

3 writing surface. 

1 38. The method of claim 27 in which light sources on the 

2 writing instrument are operated to enabling filtering of noise and 

3 interference. 
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1 39. The method of claim 27 in which light sources on the 

2 writing instrument are sequenced to encode functionality that 

3 includes at least one of erasing or biometrics of handwriting. 

1 40. The method of claim 27 in which there are enough light 

2 sources associated with the writing instrument to prevent loss of 

3 tracking when one or more of the light sources are blocked. 

1 41. A method comprising 

2 using a two-dimensional sensor in a digital camera to 

3 permit subpixel calculations of a position of a writing instrument 

4 that is in the field of view of the camera, 

averaging lines, and 

applying an algorithm to derive a final position from the 
averaged lines. 
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